Travel assistance device and computer program

ABSTRACT

Travel assistance devices and programs obtain a captured image that captures a surrounding environment of a vehicle by an imaging device having a start point of a driver&#39;s line of sight and an optical axis corresponding to a driver&#39;s line-of-sight direction. The devices and programs identify, based on the captured image and map information of an area around the vehicle, a location where a connecting road is present in the captured image. The connecting road is a road connecting to a traveling road of the vehicle at a divergence point present around the vehicle. The devices and programs determine driver&#39;s visibility of the connecting road based on a positional relationship between the location of the connecting road and a location of an obstacle present around the vehicle in the captured image and output a result of the determination.

TECHNICAL FIELD

Related technical fields include travel assistance devices and computerprograms that provide vehicle travel assistance.

BACKGROUND

In recent years, for example, as one type of travel assistance for avehicle, travel assistance has been performed in which risk factorspresent around a mobile unit are determined and guidance, vehiclecontrol, or the like, based on a result of the determination isperformed. The risk factors are factors to which attention is to be paidwhen the mobile unit travels. Particularly, one of the risk factors is aroad (hereinafter, referred to as connecting road) that connects to atraveling road of a vehicle at a divergence point present around thevehicle. The connecting road is a road having another vehicle, apedestrian, a bicycle, etc., moving thereon and possibly entering thetraveling road of the vehicle in the future. Thus, it is very importantfor a driver to grasp the presence of the connecting road to payattention beforehand to such an entry of another vehicle, a pedestrian,a bicycle, etc.

Note, however, that even if there is a connecting road, if theconnecting road has no obstacles therearound and thus can be clearlyvisually identified by the driver, then even if guidance on theconnecting road has not been provided, the driver can clearly grasp thepresence of the connecting road and thus there is very little necessityto provide assistance (e.g., guidance or vehicle control) targeting onthe connecting road. On the other hand, it is desirable that aconnecting road that is difficult for the driver to visually identifydue to obstacles therearound serve as an assistance target. Therefore,to provide appropriate travel assistance to the driver, it is importantto determine driver's visibility of a connecting road. In view of this,for example, JP 2012-192878 A discloses that during traveling of avehicle, the locations and movement speeds of obstacles such as othervehicles present around the vehicle are detected in real time using acamera, a sensor, and a communication device installed on the vehicle,and when it is determined that a vehicle's blind spot region includes aconnecting road and the risk level of the blind spot region is high,driving assistance to avoid another vehicle located on the connectingroad is provided.

SUMMARY

However, in the above-described JP 2012-192878 A, though a blind spotregion is determined in a bird's-eye view manner based on a positionalrelationship between a vehicle and another vehicle, a surroundingenvironment that can be actually visually identified by a driver is notconsidered, and thus, there has been a case in which a region determinedto be a blind spot region differs from an actual driver's blind spotregion. Therefore, a connecting road that is not actually included in ablind spot region is also determined to be a risk factor and thus mayserve as an assistance target.

Exemplary embodiments of the broad inventive principles described hereinsolve the above-described conventional problem, and provide a travelassistance device and a computer program that allow to provideappropriate travel assistance for a connecting road by determiningdriver's visibility of the connecting road and outputting a result ofthe determination.

Exemplary embodiments provide travel assistance devices and programsobtain a captured image that captures a surrounding environment of avehicle by an imaging device having a start point of a driver's line ofsight and an optical axis corresponding to a driver's line-of-sightdirection. The devices and programs identify, based on the capturedimage and map information of an area around the vehicle, a locationwhere a connecting road is present in the captured image. The connectingroad is a road connecting to a traveling road of the vehicle at adivergence point present around the vehicle. The devices and programsdetermine driver's visibility of the connecting road based on apositional relationship between the location of the connecting road anda location of an obstacle present around the vehicle in the capturedimage and output a result of the determination.

Note that the “visibility” may simply indicate only whether the drivercan see the connecting road, and may also indicate the easiness to seethe connecting road from the driver or the easiness to grasp the stateof the connecting road by the driver. The above-described “visibility”is identified using an index for determining, for example, whether atleast a part of the connecting road can be seen or whether a specificarea of the connecting road can be seen, in addition to an index fordetermining whether the entire connecting road can be seen.

Note also that the expression “outputting a result of the determination”also includes performing vehicle control based on the result of thedetermination, in addition to providing guidance on the result of thedetermination.

According to the travel assistance device and computer program that havethe above-described configurations, by determining driver's visibilityof a connecting road and outputting a result of the determination, itbecomes possible to provide appropriate travel assistance for theconnecting road. For example, it becomes possible to limit unnecessaryguidance or vehicle control as much as possible while providing travelassistance, such as guidance or vehicle control, for a connecting roadthat has low visibility and can possibly become a risk factor.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a navigation device according to thepresent embodiment.

FIG. 2 is a flowchart of a travel assistance processing programaccording to the present embodiment.

FIG. 3 is a diagram showing an image capturing range of a capturedimage.

FIG. 4 is a diagram for comparison between a captured image and a mapinformation image.

FIG. 5 is a diagram showing a determination region set on the capturedimage.

FIG. 6 is a diagram showing exemplary calculation of an overlappingpercentage of the determination region and an obstacle.

FIG. 7 is a diagram showing an example of division of the determinationregion.

FIG. 8 is a diagram showing examples of division of the determinationregion.

FIG. 9 is a diagram showing an example of a superimposed image thatprovides guidance on an invisible road.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

A travel assistance device will be described in detail below based onone embodiment that embodies a navigation device and with reference tothe drawings. First, a schematic configuration of a navigation device 1according to the present embodiment will be described using FIG. 1. FIG.1 is a block diagram showing the navigation device 1 according to thepresent embodiment.

As shown in FIG. 1, the navigation device 1 according to the presentembodiment includes a current location detecting part 11 that detects acurrent location of a vehicle having the navigation device 1 mountedthereon; a data recording part 12 having various types of data recordedtherein; a navigation ECU 13 that performs various types of arithmeticprocessing based on inputted information; an operating part 14 thataccepts operations from a user; a liquid crystal display 15 thatdisplays a map of an area around the vehicle, information about a guidedroute set on the navigation device 1, etc., to the user; a speaker 16that outputs audio guidance on route guidance, an alert against riskfactors, etc.; a DVD drive 17 that reads a DVD which is a storagemedium; and a communication module 18 that performs communication withinformation centers such as a probe center and a VICS (registeredtrademark: Vehicle Information and Communication System) center. As usedherein, the term “storage medium” does not encompass transitory signals.In addition, a HUD (head-up display) 19 and an exterior camera 20 areconnected to the navigation device 1 through an in-vehicle network suchas a CAN.

Each component included in the navigation device 1 will be describedbelow in turn.

The current location detecting part 11 includes a GPS 21, a vehiclespeed sensor 22, a steering sensor 23, a gyro sensor 24, etc., and candetect the current location, orientation, and travel speed of thevehicle, the current time, etc. Here, particularly, the vehicle speedsensor 22 is a sensor for detecting the movement distance and vehiclespeed of the vehicle, and generates pulses according to the rotation ofdrive wheels of the vehicle and outputs a pulse signal to the navigationECU 13. Then, the navigation ECU 13 counts the generated pulses andthereby calculates the rotational speed of the drive wheels and amovement distance. Note that the navigation device 1 does not need toinclude all of the above-described four types of sensors and may beconfigured to include only one or a plurality of types of sensors amongthose sensors.

In addition, the data recording part 12 includes a hard disk (not shown)serving as an external storage device and a recording medium; and arecording head (not shown) which is a driver for reading a mapinformation DB 31, a captured-image DB 32, a predetermined program,etc., recorded on the hard disk, and writing predetermined data to thehard disk. Note that the data recording part 12 may include a memorycard or an optical disc such as a CD or a DVD instead of the hard disk.Note also that the map information DB 31 and the captured-image DB 32may be stored on an external server, and the navigation device 1 mayobtain the map information DB 31 and the captured-image DB 32 bycommunication.

Here, the map information DB 31 stores therein each of two-dimensionalmap information 33 and a three-dimensional map information 34. Thetwo-dimensional map information 33 is general map information used inthe navigation device 1 and includes, for example, link data about roads(links), node data about node points, facility data about facilities,search data used in a route search process, map display data fordisplaying a map, intersection data about each intersection, andretrieval data for retrieving points.

On the other hand, the three-dimensional map information 34 isinformation about a map image that three-dimensionally represents a map.Particularly, in the present embodiment, the three-dimensional mapinformation 34 is information about a map image that three-dimensionallyrepresents road outlines. Note that the map image may also representother information than road outlines. For example, the map image mayalso three-dimensionally represent the shapes of facilities, the sectionlines of roads, road signs, signs, etc.

The navigation device 1 performs general functions such as display of amap image on the liquid crystal display 15 and a search for a guidedroute, using the two-dimensional map information 33. In addition, aswill be described later, a process related to a determination ofvisibility of a connecting road is performed using the three-dimensionalmap information 34.

In addition, the captured-image DB 32 is storage means in which capturedimages 35 captured by the exterior camera 20 are stored. Note that thecaptured images 35 captured by the exterior camera 20 are cumulativelystored in the captured-image DB 32 and deleted in turn from the oldones.

Meanwhile, the navigation ECU (electronic control unit) 13 is anelectronic control unit that performs overall control of the navigationdevice 1, and includes a CPU 41 serving as a computing device and acontrol device; and internal storage devices such as a RAM 42 that isused as a working memory when the CPU 41 performs various types ofarithmetic processing and that stores route data obtained when a routeis searched for, etc., a ROM 43 having recorded therein a travelassistance processing program (see FIG. 2) which will be describedlater, etc., in addition to a program for control, and a flash memory 44that stores a program read from the ROM 43. Note that the navigation ECU13 includes various types of means serving as processing algorithms. Forexample, surrounding environment imaging means obtains a captured imagethat captures the surrounding environment of the vehicle by the exteriorcamera 20 having a start point of a driver's line of sight and anoptical axis corresponding to a driver's line-of-sight direction.Connecting road identifying means identifies, based on the capturedimage and map information of an area around the vehicle, a locationwhere a connecting road is present in the captured image, the connectingroad being a road connecting to a traveling road of the vehicle at adivergence point present around the vehicle. Visibility determiningmeans determines driver's visibility of the connecting road, based on apositional relationship between the location of the connecting road anda location of an obstacle present around the vehicle in the capturedimage. Output means outputs a result of the determination made by thevisibility determining means.

The operating part 14 is operated when, for example, a point ofdeparture serving as a travel start point and a destination serving as atravel end point are inputted, and includes a plurality of operatingswitches such as various types of keys and buttons (not shown). Based onswitch signals outputted by, for example, pressing each switch, thenavigation ECU 13 performs control to perform corresponding varioustypes of operation. Note that the operating part 14 may include a touchpanel provided on the front of the liquid crystal display 15. Note alsothat the operating part 14 may include a microphone and an audiorecognition device.

In addition, on the liquid crystal display 15 there are displayed a mapimage including roads, traffic information, operation guidance, anoperation menu, guidance on keys, a guided route set on the navigationdevice 1, guidance information according to the guided route, news, aweather forecast, time, an e-mail, a TV program, etc. Note that in thepresent embodiment since the HUD 19 is provided as information displaymeans, if the HUD 19 is configured to also perform the above-describeddisplay of a map image, etc., then the liquid crystal display 15 may beomitted.

In addition, the speaker 16 outputs audio guidance that providesguidance on travel along a guided route or guidance on trafficinformation, based on an instruction from the navigation ECU 13. Inaddition, in the present embodiment, particularly, guidance on aconnecting road determined to have visibility lower than the thresholdis also outputted.

In addition, the DVD drive 17 is a drive that can read data recorded ona recording medium such as a DVD or a CD. Then, based on the read data,for example, music or video is played back or the map information DB 31is updated. Note that a card slot for performing reading and writing ona memory card may be provided instead of the DVD drive 17.

In addition, the communication module 18 is a communication device forreceiving traffic information transmitted from traffic informationcenters, e.g., a VICS center and a probe center, and corresponds, forexample, to a mobile phone or a DCM.

Meanwhile, the HUD 19 is placed in a vehicle's dashboard, and includestherein a liquid crystal display or a screen which is a video displaysurface on which video is displayed. The HUD 19 is configured such thatthe video displayed on the liquid crystal display or screen is furtherreflected onto a windshield in front of a driver's seat through aconcave mirror, etc., included in the HUD 19, by which a vehicle'soccupant visually identifies the video. Note that it is configured suchthat when the occupant visually identifies the video displayed on theliquid crystal display or screen and reflected onto the windshield, thevideo displayed on the liquid crystal display or screen is visuallyidentified by the occupant as a virtual image in a position far ahead ofthe windshield instead of the position of the windshield. As a result,it becomes possible to allow the occupant to visually identify thevirtual image superimposed on the real view of the environment ahead.

Note that the video displayed on the liquid crystal display or screenincludes information about the vehicle and various types of informationused to assist in occupant's driving. Particularly, in the presentembodiment, information for informing the driver of a connecting roaddetermined to have visibility lower than the threshold is displayed.

In addition, although in the present embodiment the HUD 19 is used asmeans for displaying an image superimposed on the real view of theenvironment ahead, other means may be used. For example, a windshielddisplay (WSD) that displays video on the windshield may be used. Forexample, video may be displayed from a projector using the windshield asa screen, or the windshield may be a transmissive liquid crystaldisplay. An image displayed on the windshield by the WSD is an imagesuperimposed on the real view of the environment ahead as with the HUD19.

Meanwhile, the exterior camera 20 is composed of, for example, a camerausing a solid-state imaging device such as a CCD, and is attached to theback of a vehicle's rearview mirror, a vehicle's front bumper, etc., andis placed such that an optical-axis direction is downward at apredetermined angle relative to the horizontal. The exterior camera 20captures an image of the surrounding environment ahead in a vehicle'straveling direction. In addition, the navigation ECU 13, as will bedescribed later, determines the visibility of a connecting road presentaround the vehicle by comparing a captured image having been capturedwith an image of the three-dimensional map information 34. Note that theexterior camera 20 may be configured to be disposed on the side or rearof the vehicle, too. In addition, it is desirable to make an adjustmentsuch that the placement position of the exterior camera 20 issubstantially the same as a driver's eye position (a start point of theline of sight) and the optical-axis direction is substantially the sameas a driver's line-of-sight direction obtained at normal times. By doingso, an image captured by the exterior camera 20 matches the driver'sfield of vision, enabling to more appropriately determine the visibilityof a connecting road.

Note that, in the present embodiment, the visibility of a connectingroad that is present in the driver's field of vision and that connectsto a traveling road of the vehicle at a divergence point present aroundthe vehicle is determined particularly based on a captured image of theexterior camera 20 and the three-dimensional map information 34. Theabove-described connecting road is a road having another vehicle, apedestrian, a bicycle, etc., moving thereon and possibly entering thetraveling road of the vehicle in the future. Hence, as will be describedlater, when the driver has low visibility of the connecting road (i.e.,it is estimated that the driver has not been able to grasp theconnecting road), guidance for allowing the driver to grasp the presenceof the connecting road is provided.

Next, a travel assistance processing program executed by the CPU 41 inthe navigation device 1 according to the present embodiment that has theabove-described configuration will be described based on FIG. 2. FIG. 2is a flowchart of the travel assistance processing program according tothe present embodiment. Here, the travel assistance processing programis a program that is executed after turning on a vehicle's ACC(accessory) power supply, and determines the driver's visibility of aconnecting road based on a captured image captured by the exteriorcamera 20 and the three-dimensional map information 34 and providesguidance on a connecting road with low visibility. In addition, thefollowing program shown in the flowchart of FIG. 2 is stored in the RAM42, the ROM 43, etc., included in the navigation ECU 13, and executed bythe CPU 41.

First, in the travel assistance processing program, at step(hereinafter, abbreviated as S) 1, the CPU 41 obtains a vehicle'scurrent location and orientation based on results of detection by thecurrent location detecting part 11. Specifically, positional coordinateson a map that indicate a vehicle's current location are obtained usingthe two-dimensional map information 33. Note that upon detection of avehicle's current location, a map-matching process for matching thevehicle's current location to the two-dimensional map information 33 isalso performed. Furthermore, the vehicle's current location may beidentified using a high-accuracy location technique. Here, thehigh-accuracy location technique is a technique allowing to detect atravel lane or a high-accuracy vehicle location by detecting, by imagerecognition, white line and road surface painting information capturedfrom a camera placed on the vehicle and further checking the white lineand road surface painting information against a map information DBstored in advance. Note that the details of the high-accuracy locationtechnique are already publicly known and thus are omitted. Note that itis desirable that the vehicle's current location and orientation beultimately identified on a map of the three-dimensional map information34, too.

Then, at S2, the CPU 41 obtains a captured image captured recently bythe exterior camera 20 from the captured-image DB 32. Note that thecaptured image captured by the exterior camera 20 is an image thatcaptures the environment ahead in a vehicle's traveling direction, i.e.,the environment ahead visually identified by the driver (driver's fieldof vision), to correspond to a start point of a driver's line of sight(eye point) and a driver's line-of-sight direction.

Subsequently, at S3, the CPU 41 obtains particularly three-dimensionalmap information 34 for an area around the vehicle's current locationwhich is identified at the above-described 51 (e.g., an area within 300m from the vehicle's current location) among the three-dimensional mapinformation 34 stored in the map information DB 31.

Thereafter, at S4, the CPU 41 obtains an image capturing range of thecaptured image obtained at the above-described S2. Here, as shown inFIG. 3, the image capturing range of a captured image 35 can beidentified by the position of a focal point P, an optical-axis directionα, and the angle of view ϕ of the exterior camera 20 obtained at thepoint in time of image capturing. Note that the angle of view ϕ is afixed value which is determined in advance by the exterior camera 20. Onthe other hand, the position of the focal point P is determined based onthe vehicle's current location obtained at the above-described S1 andthe placement position of the exterior camera 20 in the vehicle. Inaddition, the optical-axis direction α is determined based on thevehicle's orientation obtained at the above-described S1 and theplacement direction of the exterior camera 20 in the vehicle.

Then, at S5, the CPU 41 creates a bird's-eye-view image (hereinafter,referred to as a map information image) which three-dimensionallyrepresents a map of the same range as the image capturing range of thecaptured image obtained at the above-described S4 from the samedirection as an image capturing direction of the captured image, usingthe three-dimensional map information 34 obtained at the above-describedS3. Note that the map information image itself is a two-dimensionalimage which is the same as the captured image.

Here, FIG. 4 is a diagram showing an example of a map information image52 created for a captured image 51. As shown in FIG. 4, the mapinformation image 52 is an image in which lines indicating outlines 53of roads included in an image capturing range of the captured image 51(i.e., in the driver's field of vision) are drawn. In the mapinformation image 52 there is also drawn an outline of a road which ishidden in the captured image 51 by obstacles such as other vehicles.

Subsequently, at S6, the CPU 41 detects obstacles located around thevehicle, by performing image recognition processes on the captured imageobtained at the above-described S2. For the image recognition processes,a binarization process, a pattern matching process using feature pointsand a template, etc., are performed, but since those image recognitionprocesses are already publicly known, the details thereof are omitted.In addition, at the above-described S3, each of the “type of anobstacle,” “shape (detected range),” “distance from the vehicle,” and“positional coordinates” is also obtained. The “type of an obstacle”includes, for example, an automobile, a pedestrian, and a bicycle. The“positional coordinates” are positional coordinates on the capturedimage with a rectangular coordinate system being set on the capturedimage. In addition, for a method of detecting an obstacle, detection maybe performed using a sensor placed on the vehicle or information aboutan obstacle may be obtained from an external source by communication, inaddition to detection from the captured image of the exterior camera 20.Note that an obstacle does not necessarily need to be one having acertain shape, and as long as the obstacle is an object that impedes thevisibility of a connecting road, the obstacle may be, for example, lightor darkness.

For example, in the example shown in FIG. 4, three vehicles 54 to 56 aredetected as obstacles, and the “type of an obstacle,” “shape (detectedrange),” “distance from the vehicle,” and “positional coordinates” areidentified for each of the vehicles 54 to 56.

Furthermore, at S7, the CPU 41 identifies the orientation (travelingdirection) of each obstacle detected at the above-described S6 and alocation where the obstacle is detected. Note that the “orientation(traveling direction) of the obstacle” may be identified from the typeand shape of the obstacle, or may be identified based on changes in thelocation of the obstacle in captured images which are consecutivelycaptured at a predetermined interval. In addition, for the “locationwhere the obstacle is detected,” it is identified, based on thepositional coordinates of the obstacles detected at the above-describedS6 and the map information image created at the above-described S5,whether each obstacle is located on a road or located on other than aroad. Furthermore, when the obstacle is on a road, it is desirable toalso identify whether the obstacle is located on a lane where thevehicle travels, or located on an opposite lane, or located on aconnecting road. In addition, the above-described S7 is performedtargeting on all obstacles detected at the above-described S6.

For example, in the example shown in FIG. 4, for the three vehicles 54to 56, it is identified that the “orientation (traveling direction) ofthe obstacle” is backward (the opposite direction to the orientation ofthe vehicle). In addition, the “location where the obstacle is detected”is identified to be on an opposite lane.

Then, at S8, the CPU 41 sets a determination region for determining thevisibility of a connecting road present in the driver's field of vision,on the captured image obtained at the above-described S2. Note that the‘connecting road’ is a road that connects to a traveling road of thevehicle at a divergence point present around the vehicle. For example,in the example shown in FIG. 4, roads 58 and 59 connecting to adivergence point 57 correspond to connecting roads. Note that althoughin the present embodiment, particularly, the roads 58 and 59 connectingin a direction intersecting the traveling road of the vehicle serve asconnecting roads, a road 60 connecting in the traveling direction mayalso be included in connecting roads. In addition, when there are aplurality of connecting roads, a determination region is set for eachconnecting road. Note that for a case in which there are no connectingroads in the captured image obtained at the above-described S2 (i.e., inthe driver's field of vision) (a case in which a connecting road ishidden by obstacles, etc., and cannot be seen is also considered thatthere is a connecting road), the travel assistance process ends withoutperforming processes at and after S8.

A method of setting a determination region at the above-described S8will be described in more detail below using FIG. 5. The followingdescription is made using, as an example, a determination region whichis set for the connecting road 59 connecting in a right direction to thedivergence point 57 present ahead in the vehicle's traveling direction.

First, the CPU 41 identifies a location where the connecting road 59 ispresent in the captured image 51, based on the map information image 52created at the above-described S5. Specifically, an area in the capturedimage 51 corresponding to an area enclosed by an outline 53 of theconnecting road 59 in the map information image 52 is identified. Theidentified area is a location where the connecting road 59 is present inthe captured image 51. For example, in FIG. 5, a hatched region is alocation where the connecting road 59 is present in the captured image51. Subsequently, the CPU 41 identifies an end portion X in the capturedimage 51 at which the connecting road 59 connects to the divergencepoint 57 and is located nearest to the vehicle side, based on theidentified location of the connecting road 59. Then, a rectangularregion of 3 m in a direction going away from the connecting divergencepoint 57 along the connecting road 59 and 2 m in a height direction withthe end portion X identified in the captured image 51 being a point oforigin is set as a determination region 61. Likewise, a determinationregion is also set for other connecting roads.

Note that the determination region 61 set at the above-described S8targets on the connecting road 59 present in the captured image 51,i.e., in the driver's field of vision, and is a region including atleast a part of the connecting road 59. More specifically, thedetermination region 61 is set in a region including a road portionwithin 3 mm along the connecting road 59 from an end portion at whichthe connecting road 59 connects to the divergence point 57.

Then, at S9, the CPU 41 determines, based on a positional relationshipbetween the location of a connecting road in the captured image obtainedat the above-described S2 and the locations of the obstacles in the samecaptured image, the driver's visibility of the connecting road presentin the captured image (i.e., in the driver's field of vision). Note thatwhen there are a plurality of connecting roads, visibility is determinedfor each connecting road. Particularly, in the present embodiment, basedon a positional relationship between the connecting road in thedetermination region set at the above-described S8 and the obstacles,the driver's visibility of the connecting road is determined.

A method of determining visibility at the above-described S9 will bedescribed in more detail below using FIG. 6. The following descriptionis made using, as an example, determination of visibility of theconnecting road 59 connecting in the right direction to the divergencepoint 57 present ahead in the vehicle's traveling direction.

First, the CPU 41 identifies locations where the obstacles detected atthe above-described S6 (in the example shown in FIG. 6, the vehicles 54to 56) are present in the captured image 51. Then, an overlappingpercentage of the vehicles 54 to 56 overlapping the determination region61 is calculated. Specifically, as shown in FIG. 6, a percentage of anoverlapping area of the vehicles 54 to 56 in the area of the entiredetermination region 61 is calculated as an overlapping percentage. Notethat the locations and shapes of the vehicles 54 to 56 are detected atthe above-described S6. Then, the calculated overlapping percentage isset as a numerical value indicating the driver's visibility of theconnecting road 59. Note that when the calculated overlapping percentage(visibility) is greater than or equal to a threshold (e.g., 80%), theCPU 41 determines that the corresponding connecting road cannot bevisually identified by the driver, and when the overlapping percentageis less than the threshold, the CPU 41 determines that the correspondingconnecting road can be visually identified by the driver.

In addition, as shown in FIG. 8, the determination region 61 may bedivided into a plurality of sections. For example, FIG. 8 shows anexample in which the determination region 61 is divided into foursections 71 to 74 with lines connecting the midpoints of opposite sidesbeing boundaries. Furthermore, in the present embodiment, as shown inthe following (A) and (B), it is also possible to set differentvisibility calculation criteria for the different sections 71 to 74.Note that the following description is made with an upper left sectionbeing a section 71, a lower left section being a section 72, an upperright section being a section 73, and a lower right section being asection 74.

(A) As an example, such calculation criteria are set that a higheroverlapping percentage is likely to be calculated for a section closerfrom the end portion connecting to the divergence point among theplurality of sections 71 to 74. Here, since the section closer from theend portion connecting to the divergence point is a section located nearan entrance to the connecting road, the section is an important sectionparticularly when the driver determines whether there is a vehicle or apedestrian entering the traveling road of the vehicle from theconnecting road. Thus, such calculation criteria are set that a higheroverlapping percentage is likely to be calculated for a section closerfrom the end portion connecting to the divergence point.

For example, for the determination region 61 set for the connecting road59 connecting in the right direction to the divergence point 57 presentahead in the vehicle's traveling direction as shown in FIG. 5, suchcalculation criteria are set that a higher overlapping percentage islikely to be calculated for the sections 71 and 72 close to thedivergence point 57 than for the sections 73 and 74. For example, forthe sections 71 and 72, a percentage obtained by multiplying apercentage of obstacles actually overlapping the sections 71 and 72 by afactor of 1.2 is calculated as an overlapping percentage. On the otherhand, for the sections 73 and 74, a percentage obtained by multiplying apercentage of obstacles actually overlapping the sections 73 and 74 by afactor of 0.8 is calculated as an overlapping percentage. Then, a valueobtained by totaling the overlapping percentages calculated for eachsection is determined to be an overlapping percentage of the obstaclesoverlapping the determination region 61.

Note that a determination region set for a connecting road connecting tothe divergence point in a left direction is left-right reversed withrespect to the above-described example.

(B) As another example, such calculation criteria are set that a higheroverlapping percentage is likely to be calculated for a section more onthe lower side among the plurality of sections 71 to 74. Here, since, asshown in FIG. 5, a lower edge of the determination region 61 is setbased on the end portion X on a side of the connecting road 59 near thevehicle, the percentage of overlap with the connecting road 59 is higherfor the sections 72 and 74 on the lower side than for the sections 71and 73 on the upper side. Namely, since a section on the lower side is asection where a vehicle or a pedestrian moving on the connecting road ishighly likely to be located, the section is an important sectionparticularly when the driver determines whether there is a vehicle or apedestrian entering the traveling road of the vehicle from theconnecting road. Therefore, such calculation criteria are set that ahigher overlapping percentage is likely to be calculated for a sectionmore on the lower side.

For example, for the sections 72 and 74, a percentage obtained bymultiplying a percentage of obstacles actually overlapping the sections72 and 74 by a factor of 1.2 is calculated as an overlapping percentage.On the other hand, for the sections 71 and 73, a percentage obtained bymultiplying a percentage of obstacles actually overlapping the sections71 and 73 by a factor of 0.8 is calculated as an overlapping percentage.Then, a value obtained by totaling the overlapping percentagescalculated for each section is set as an overlapping percentage of theobstacles overlapping the determination region 61.

Note that the above-described (A) and (B) may be combined. In that case,for the sections 71, 72, and 74, a percentage obtained by multiplying apercentage of obstacles actually overlapping the sections 71, 72, and 74by a factor of 1.2 is calculated as an overlapping percentage. On theother hand, for the section 73, a percentage obtained by multiplying apercentage of obstacles actually overlapping the section 73 by a factorof 0.8 is calculated as an overlapping percentage. Then, a valueobtained by totaling the overlapping percentages calculated for eachblock is set as an overlapping percentage of the obstacles overlappingthe determination region 61. Furthermore, for the section 72, themultiplication factor may be 1.4.

In addition, the boundaries that divide the determination region 61 maybe changed according to the road width of the connecting road.Specifically, as shown in FIG. 8, when the road width of the connectingroad is narrow, a boundary that divides the upper and lower portions isallowed to move to the lower side, and when the road width of theconnecting road is wide, the boundary that divides the upper and lowerportions is allowed to move to the upper side. By doing so, anoverlapping percentage can be more appropriately corrected in theabove-described (B).

Then, at S10, the CPU 41 determines whether there is a connecting roadwhich is present within a stoppable distance from the vehicle and whoseoverlapping percentage (visibility) calculated at the above-described S9is greater than or equal to the threshold (e.g., 80%), i.e., which ispresent in the driver's field of vision and cannot be visuallyidentified by the driver. Note that the ‘stoppable distance’ is adistance within which the vehicle can stop at predetermined acceleration(with an upper limit that does not place strain on the driver) or less,and is calculated based on the current vehicle speed of the vehicle. Inaddition, the threshold serving as a determination criterion at theabove-described S10 can be changed as appropriate and may be, forexample, 50%. In addition, the threshold may be allowed to be changeddepending on the surrounding environment. For example, a higherthreshold is set for a poor visibility environment such as at night orin rain than for other environments.

Then, if it is determined that there is a connecting road which ispresent within a stoppable distance from the vehicle and whoseoverlapping percentage (visibility) calculated at the above-described S9is greater than or equal to the threshold (e.g., 80%), i.e., whichcannot be visually identified by the driver (S10: YES), processingtransitions to S11. On the other hand, if it is determined that there isno connecting road whose overlapping percentage (visibility) calculatedat the above-described S9 is greater than or equal to the threshold(e.g., 80%) or that, even if there is such a connecting road, theconnecting road is out of the stoppable distance (S10: NO), processingtransitions to S13.

At S11, the CPU 41 determines whether the obstacles overlapping theconnecting road are vehicles and the vehicles are located on theconnecting road, based on the types of the obstacles identified at theabove-described S6 and the orientations (traveling directions) anddetection locations of the obstacles identified at the above-describedS7. Here, even if the connecting road cannot be visually identified dueto another vehicle overlapping the connecting road, if the other vehicleis a vehicle traveling on the connecting road, then the driver can graspthe location and shape of the connecting road to some extent from thelocation of the other vehicle, and can also determine whether there is avehicle entering the traveling road of the vehicle from the connectingroad. Thus, in such a case, even if the connecting road has visibilityless than or equal to the threshold, the driver is exceptionallyconsidered to be able to visually identify the connecting road.

Then, if it is determined that the obstacles overlapping the connectingroad are vehicles and the vehicles are located on the connecting road(S11: YES), processing transitions to S13. On the other hand, if it isdetermined that the obstacles overlapping the connecting road are otherthan vehicles or that, even if the obstacles are vehicles, the vehiclesare not located on the connecting road (S11: NO), processing transitionsto S12.

At S12, the CPU 41 transmits a control signal to the HUD 19, and outputsvideo informing of the presence of the connecting road that isdetermined to be not visually identifiable by the driver (hereinafter,referred to as invisible road) to the liquid crystal display of the HUD19. Specifically, a superimposed image which is superimposed on a rangeof the invisible road that includes the end portion connecting to thedivergence point, and visually identified by the driver is displayed. Inaddition, a sound alerting the driver to the invisible road may beoutputted from the speaker 16.

For example, FIG. 9 is a diagram showing an example of a superimposedimage displayed at the above-described S12.

As shown in FIG. 9, a virtual image (superimposed image) 75 of anexclamation mark informing of the presence of an invisible road isdisplayed in a position that is a predetermined distance (25 m) ahead inthe vehicle's traveling direction. Particularly, since the virtual image75 is superimposed on a range of the invisible road that includes theend portion connecting to the divergence point, i.e., an area near theentrance to the invisible road, and visually identified by the driver,even if there is another vehicle or a pedestrian entering the travelingroad of the vehicle from the invisible road, the driver can payattention beforehand to the entry of another vehicle or a pedestrian.

In addition, it is desirable that the size of the superimposed imagedisplayed at the above-described S12 increases as the distance from thevehicle to the invisible road decreases. By doing so, it becomespossible to allow the driver to visually grasp he distance to theinvisible road.

On the other hand, at S13, the CPU 41 determines that there is noconnecting road in the captured image (i.e., in the driver's field ofvision) that cannot be visually identified by the driver, and thus endsthe travel assistance process without providing any particular guidanceon a connecting road.

Note that in the present embodiment, as output means for outputting aresult of a determination made for the driver's visibility of aconnecting road, particularly, guidance informing the driver of thepresence of an invisible road determined to be not visually identifiableby the driver (the visibility is less than or equal to the threshold) isprovided, but as the output means for outputting a result of adetermination, vehicle control based on the result of a determinationmay be performed. For example, it is possible to perform vehicle controlfor allowing the driver to notice an invisible road, or vehicle controlto avoid an invisible road. Specifically, there is vehicle control thatperforms deceleration just before an invisible road. In addition, it isalso possible to apply a result of a determination to a self-drivingvehicle.

Furthermore, if machine learning can determine what risk factors arepresent on an invisible road, then guidance that more specificallyidentifies a risk factor (e.g., “pay attention to a pedestrian enteringfrom a crossroad present in a blind spot ahead of the vehicle”) may beprovided.

As described in detail above, the navigation device 1 and a computerprogram executed by the navigation device 1 according to the presentembodiment obtain a captured image that captures the surroundingenvironment of a vehicle (S2); obtain a map information image which is amap image that three-dimensionally represents a map and that representsthe same range as an image capturing range of the captured image fromthe same direction as an image capturing direction of the captured image(S5); determine, based on the captured image and the map informationimage, driver's visibility of a connecting road which is a roadconnecting to a traveling road of the vehicle at a divergence pointpresent around the vehicle (S10); and output a result of the visibilitydetermination (S12). Thus, it becomes possible to provide appropriatetravel assistance for the connecting road. For example, it becomespossible to limit unnecessary guidance or vehicle control as much aspossible while providing travel assistance, such as guidance or vehiclecontrol, for a connecting road that has low visibility and can possiblybecome a risk factor.

Note that various improvements and modifications may, of course, be madewithout departing from the broad inventive principles.

For example, although in the present embodiment the three-dimensionalmap information 34 is information about a map image thatthree-dimensionally represents road outlines, the three-dimensional mapinformation 34 may be map information representing information otherthan road outlines. For example, the map image may also represent theshapes of facilities, the section lines of roads, road signs, signs,etc.

In addition, although in the present embodiment, as guidance informingthe driver of an invisible road, an image superimposed on an area nearan entrance to the invisible road and visually identified is displayed(FIG. 9), the superimposed image may be an image in other modes. Forexample, an image of line segments superimposed on an outline of aninvisible road and visually identified may be displayed.

In addition, although in the present embodiment the determination region61 has a rectangular shape which is 3 m wide and 2 m long, the size andshape of the determination region 61 can be changed as appropriate, andfor example, a circular shape may be adopted. In addition, although FIG.7 shows an example in which the determination region 61 is divided intofour sections, the number of divisions may be three or less, or five ormore.

In addition, although in the present embodiment the travel assistanceprocessing program (FIG. 2) is executed by the navigation device 1, thetravel assistance processing program may be configured to be executed byan in-vehicle device other than the navigation device. For example, thetravel assistance processing program may be configured to be executed bya control part of the HUD 19, a vehicle control ECU, or other in-vehicledevices. Note that when the control part of the HUD 19 executes thetravel assistance processing program, the travel assistance device canalso be configured by the HUD 19. In addition, instead of an in-vehicledevice performing all processes, an external server may perform some ofthe processes.

In addition, although an implementation example in which the travelassistance device is embodied is described above, the travel assistancedevice can also have the following configurations, and in that case, thefollowing advantageous effects are provided.

For example, a first configuration is as follows:

A travel assistance device includes surrounding environment imagingmeans (41) for obtaining a captured image that captures the surroundingenvironment of a vehicle by an imaging device (20) having a start pointof a driver's line of sight and an optical axis corresponding to adriver's line-of-sight direction; connecting road identifying means (41)for identifying, based on the captured image and map information of anarea around the vehicle, a location where a connecting road is presentin the captured image, the connecting road being a road connecting to atraveling road of the vehicle at a divergence point present around thevehicle; visibility determining means (41) for determining driver'svisibility of the connecting road, based on a positional relationshipbetween the location of the connecting road and a location of anobstacle (54 to 56) present around the vehicle in the captured image;and output means (41) for outputting a result of the determination madeby the visibility determining means.

According to the travel assistance device having the above-describedconfiguration, by determining driver's visibility of a connecting roadand outputting a result of the determination, it becomes possible toprovide appropriate travel assistance for the connecting road. Forexample, it becomes possible to limit unnecessary guidance or vehiclecontrol as much as possible while providing travel assistance, such asguidance or vehicle control, for a connecting road that has lowvisibility and can possibly become a risk factor.

In addition, a second configuration is as follows:

The travel assistance device includes obstacle detecting means (41) fordetecting a location and a shape of the obstacle present around thevehicle from the captured image, and the visibility determining means(41) determines the driver's visibility of the connecting road, based onthe location and shape of the obstacle detected by the obstacledetecting means.

According to the travel assistance device having the above-describedconfiguration, it becomes possible to determine accurate driver'svisibility of a connecting road particularly when the visibility of theconnecting road is impeded by an obstacle present around the vehicle.

In addition, a third configuration is as follows:

The travel assistance device includes determination region setting means(41) for setting a determination region (61) in a region including atleast a part of the connecting road in the captured image, thedetermination region (61) serving as a target for determining thevisibility of the connecting road, and the visibility determining means(41) determines the driver's visibility of the connecting road, based ona positional relationship between the connecting road and the obstaclein the determination region.

According to the travel assistance device having the above-describedconfiguration, since the visibility of a connecting road is determinedbased on a positional relationship between the connecting road and anobstacle in a specific region including a part of the connecting road,it becomes possible to determine the visibility of the connecting roadconsidering particularly a region that is important when the drivergrasps the connecting road.

In addition, a fourth configuration is as follows:

The determination region setting means (41) sets the determinationregion (61) in a region including a road portion of the connecting roadwithin a predetermined distance along the connecting road from an endportion connecting to the divergence point.

According to the travel assistance device having the above-describedconfiguration, since visibility is determined targeting on particularlya range near an entrance to a connecting road, it becomes possible todetermine the visibility of the connecting road considering whether thedriver can visually identify a last-minute vehicle or pedestrianentering a traveling road of the vehicle from the connecting road.

In addition, a fifth configuration is as follows:

The visibility determining means (41) calculates an overlappingpercentage of the obstacle overlapping the determination region (61),and determines the driver's visibility of the connecting road, based onthe calculated overlapping percentage.

According to the travel assistance device having the above-describedconfiguration, it becomes possible to determine accurate driver'svisibility of a connecting road particularly when the visibility of theconnecting road is impeded by an obstacle present around the vehicle.

In addition, a sixth configuration is as follows:

The visibility determining means (41) divides the determination region(61) into a plurality of sections, and calculates the overlappingpercentage using different calculation criteria for the differentplurality of sections.

According to the travel assistance device having the above-describedconfiguration, it becomes possible to determine the visibility of aconnecting road considering particularly the visibility in a section ofthe determination region that is important when the driver grasps theconnecting road.

In addition, a seventh configuration is as follows:

The visibility determining means (41) sets such calculation criteriathat a higher overlapping percentage is likely to be calculated for asection closer from the end portion connecting to the divergence pointamong the plurality of sections.

According to the travel assistance device having the above-describedconfiguration, it becomes possible to determine the visibility of aconnecting road placing importance on particularly the visibility in asection of the determination region near an entrance to the connectingroad.

In addition, an eighth configuration is as follows:

The visibility determining means (41) sets such calculation criteriathat a higher overlapping percentage is likely to be calculated for asection more on a lower side among the plurality of sections.

According to the travel assistance device having the above-describedconfiguration, it becomes possible to determine the visibility of aconnecting road placing importance on particularly the visibility in asection of the determination region that has a high percentage ofoverlap with the connecting road.

In addition, a ninth configuration is as follows:

The visibility determining means (41) changes a boundary that dividesthe determination region (61), according to a road width of theconnecting road.

According to the travel assistance device having the above-describedconfiguration, it becomes possible to appropriately divide particularlya section of the determination region that has a high percentage ofoverlap with a connecting road, according to the road width of theconnecting road.

In addition, a tenth configuration is as follows:

The visibility determining means (41) determines that a driver cannotvisually identify the connecting road when the overlapping percentage isgreater than or equal to a threshold, and determines that the driver canvisually identify the connecting road when the overlapping percentage isless than the threshold.

According to the travel assistance device having the above-describedconfiguration, it becomes possible to accurately determine whether thedriver can visually identify a connecting road from an overlappingpercentage of an obstacle present around the vehicle and the connectingroad, particularly when the visibility of the connecting road is impededby the obstacle.

In addition, an eleventh configuration is as follows:

The visibility determining means (41) determines, when the obstacle (54to 56) is another vehicle located around the vehicle, whether the othervehicle is located on the connecting road; and determines, when it isdetermined that the other vehicle is located on the connecting road,that the driver can visually identify the connecting road, regardless ofa positional relationship between the location of the connecting roadand a location of the other vehicle in the captured image.

According to the travel assistance device having the above-describedconfiguration, even if the visibility of a connecting road is impeded byan obstacle present around the vehicle, when the obstacle is a vehiclemoving on the connecting road, it can be exceptionally determined thatthe driver can visually identify the connecting road.

1. A travel assistance device comprising: a processor programmed to:obtain a captured image that captures a surrounding environment of avehicle by an imaging device having a start point of a driver's line ofsight and an optical axis corresponding to a driver's line-of-sightdirection; identify, based on the captured image and map information ofan area around the vehicle, a location where a connecting road ispresent in the captured image, the connecting road being a roadconnecting to a traveling road of the vehicle at a divergence pointpresent around the vehicle; determine driver's visibility of theconnecting road, based on a positional relationship between the locationof the connecting road and a location of an obstacle present around thevehicle in the captured image; and output a result of the determination.2. The travel assistance device according to claim 1, wherein theprocessor is programmed to: detect a location and a shape of theobstacle present around the vehicle from the captured image; anddetermine the driver's visibility of the connecting road, based on thedetected location and shape of the obstacle.
 3. The travel assistancedevice according to claim 1, wherein the processor is programmed to seta determination region in a region including at least a part of theconnecting road in the captured image, the determination region servingas a target for determining the visibility of the connecting road; anddetermine the driver's visibility of the connecting road, based on apositional relationship between the connecting road and the obstacle inthe determination region.
 4. The travel assistance device according toclaim 3, wherein the processor is programmed to set the determinationregion in a region including a road portion of the connecting roadwithin a predetermined distance along the connecting road from an endportion connecting to the divergence point.
 5. The travel assistancedevice according to claim 3, wherein the processor is programmed to:calculate an overlapping percentage of the obstacle overlapping thedetermination region; and determine the driver's visibility of theconnecting road, based on the calculated overlapping percentage.
 6. Thetravel assistance device according to claim 5, wherein the processor isprogrammed to: divide the determination region into a plurality ofsections; and calculate the overlapping percentage using differentcalculation criteria for each of the plurality of sections.
 7. Thetravel assistance device according to claim 6, wherein the processor isprogrammed to set the calculation criteria such that a higheroverlapping percentage is likely to be calculated for a section closerfrom an end portion connecting to the divergence point among theplurality of sections.
 8. The travel assistance device according toclaim 6, wherein the processor is programmed to set the calculationcriteria such that a higher overlapping percentage is likely to becalculated for a section more on a lower side among the plurality ofsections.
 9. The travel assistance device according to claim 6, whereinthe processor is programmed to change a boundary that divides thedetermination region, according to a road width of the connecting road.10. The travel assistance device according to claim 5, wherein theprocessor is programmed to: determine that a driver cannot visuallyidentify the connecting road when the overlapping percentage is greaterthan or equal to a threshold; and determine that the driver can visuallyidentify the connecting road when the overlapping percentage is lessthan the threshold.
 11. The travel assistance device according to claim10, wherein the processor is programmed to: determine, when the obstacleis another vehicle located around the vehicle, whether the other vehicleis located on the connecting road; and determine, when it is determinedthat the other vehicle is located on the connecting road, that thedriver can visually identify the connecting road, regardless of apositional relationship between the location of the connecting road anda location of the other vehicle in the captured image.
 12. Acomputer-readable storage medium storing a computer-executable travelassistance program that causes a computer to perform functions,comprising: obtaining a captured image that captures a surroundingenvironment of a vehicle by an imaging device having a start point of adriver's line of sight and an optical axis corresponding to a driver'sline-of-sight direction; identifying, based on the captured image andmap information of an area around the vehicle, a location where aconnecting road is present in the captured image, the connecting roadbeing a road connecting to a traveling road of the vehicle at adivergence point present around the vehicle; determining driver'svisibility of the connecting road, based on a positional relationshipbetween the location of the connecting road and a location of anobstacle present around the vehicle in the captured image; andoutputting a result of the determination.